Technical Debt: An empirical investigation of its harmfulness and on management strategies in industry

Background: In order to survive in today\u27s fast-growing and ever fast-changing business environment, software companies need to continuously deliver customer value, both from a short- and long-term perspective. However, the consequences of potential long-term and far-reaching negative effects of shortcuts and quick fixes made during the software development lifecycle, described as Technical Debt (TD), can impede the software development process.Objective: The overarching goal of this Ph.D. thesis is twofold. The first goal is to empirically study and understand in what way and to what extent, TD influences today’s software development work, specifically with the intention to provide more quantitative insight into the field. Second, to understand which different initiatives can reduce the negative effects of TD and also which factors are important to consider when implementing such initiatives.Method: To achieve the objectives, a combination of both quantitative and qualitative research methodologies are used, including interviews, surveys, a systematic literature review, a longitudinal study, analysis of documents, correlation analysis, and statistical tests. In seven of the eleven studies included in this Ph.D. thesis, a combination of multiple research methods are used to achieve high validity.Results: We present results showing that software suffering from TD will cause various negative effects on both the software and the developing process. These negative effects are illustrated from a technical, financial, and a developer’s working situational perspective. These studies also identify several initiatives that can be undertaken in order to reduce the negative effects of TD.Conclusion: The results show that software developers report that they waste 23% of their working time due to experiencing TD and that TD required them to perform additional time-consuming work activities. This study also shows that, compared to all types of TD, architectural TD has the greatest negative impact on daily software development work and that TD has negative effects on several different software quality attributes. Further, the results show that TD reduces developer morale. Moreover, the findings show that intentionally introducing TD in startup companies can allow the startups to cut development time, enabling faster feedback and increased revenue, preserve resources, and decrease risk and thereby contribute to beneficial\ua0effects. This study also identifies several initiatives that can be undertaken in order to reduce the negative effects of TD, such as the introduction of a tracking process where the TD items are introduced in an official backlog. The finding also indicates that there is an unfulfilled potential regarding how managers can influence the manner in which software practitioners address TD

The use of incentives to promote Technical Debt management

When developing software, it is vitally important to keep the level of technical debt down since it is well established from several studies that technical debt can, e.g., lower the development productivity, decrease the developers' morale, and compromise the overall quality of the software. However, even if researchers and practitioners working in today's software development industry are quite familiar with the concept of technical debt and its related negative consequences, there has been no empirical research focusing specifically on how software managers actively communicate and manage the need to keep the level of technical debt as low as possible

Technical Debt Prioritization: State of the Art. A Systematic Literature Review

Background. Software companies need to manage and refactor Technical Debt issues. Therefore, it is necessary to understand if and when refactoring Technical Debt should be prioritized with respect to developing features or fixing bugs. Objective. The goal of this study is to investigate the existing body of knowledge in software engineering to understand what Technical Debt prioritization approaches have been proposed in research and industry. Method. We conducted a Systematic Literature Review among 384 unique papers published until 2018, following a consolidated methodology applied in Software Engineering. We included 38 primary studies. Results. Different approaches have been proposed for Technical Debt prioritization, all having different goals and optimizing on different criteria. The proposed measures capture only a small part of the plethora of factors used to prioritize Technical Debt qualitatively in practice. We report an impact map of such factors. However, there is a lack of empirical and validated set of tools. Conclusion. We observed that technical Debt prioritization research is preliminary and there is no consensus on what are the important factors and how to measure them. Consequently, we cannot consider current research conclusive and in this paper, we outline different directions for necessary future investigations

Measuring affective states from technical debt: A psychoempirical software engineering experiment

Software engineering is a human activity. Despite this, human aspects are under-represented in technical debt research, perhaps because they are challenging to evaluate. This study's objective was to investigate the relationship between technical debt and affective states (feelings, emotions, and moods) from software practitioners. Forty participants (N = 40) from twelve companies took part in a mixed-methods design, consisting of a repeated-measures (r = 5) experiment (n = 200), a survey employing a questionnaire, and semi-structured interviews. The statistical analysis shows that different design smells negatively or positively impact affective states. From the qualitative data, it is clear that technical debt activates a substantial portion of the emotional spectrum and is psychologically taxing. Further, the practitioner's reactions to technical debt appear to fall in different levels of maturity. We argue that human aspects in technical debt are important factors to consider, as they may result in, e.g., procrastination, apprehension, and burnout.Comment: 48 pages, 11 figures, submitted to Empirical Software Engineerin

Technical Debt Prioritization: State of the Art. A Systematic Literature Review

Background. Software companies need to manage and refactor Technical Debt issues. Therefore, it is necessary to understand if and when refactoring of Technical Debt should be prioritized with respect to developing features or fixing bugs.Objective. The goal of this study is to investigate the existing body of knowledge in software engineering to understand what Technical Debt prioritization approaches have been proposed in research and industry. Method. We conducted a Systematic Literature Review of 557 unique papers published until 2019, following a consolidated methodology applied in software engineering. We included 44 primary studies.Results. Different approaches have been proposed for Technical Debt prioritization, all having different goals and proposing optimization regarding different criteria. The proposed measures capture only a small part of the plethora of factors used to prioritize Technical Debt qualitatively in practice. We present an impact map of such factors. However, there is a lack of empirical and validated set of tools.Conclusion. We observed that Technical Debt prioritization research is preliminary and there is no consensus on what the important factors are and how to measure them. Consequently, we cannot consider current research\ua0conclusive. In this paper, we therefore outline different directions for necessary future investigations

Embracing Technical Debt, from a Startup Company Perspective

Software\ua0 startups\ua0 are\ua0 typically\ua0 under\ua0 extreme pressure to get to market quickly with limited resources and high uncertainty.\ua0 This\ua0 pressure\ua0 and\ua0 uncertainty\ua0 is\ua0 likely\ua0 to\ua0 cause startups to accumulate technical debt as they make decisions that are more focused on the short-term than the long-term health of the codebase. However, most research on technical debt has been focused\ua0 on\ua0 more\ua0 mature\ua0 software\ua0 teams,\ua0 who\ua0 may\ua0 have\ua0 less pressure\ua0 and,\ua0 therefore,\ua0 reason\ua0 about\ua0 technical\ua0 debt\ua0 very differently\ua0 than\ua0 software\ua0 startups.\ua0 In\ua0 this\ua0 study,\ua0 we\ua0 seek\ua0 to understand\ua0 the\ua0 organizational\ua0 factors\ua0 that\ua0 lead\ua0 to\ua0 and\ua0 the benefits\ua0 and\ua0 challenges\ua0 associated\ua0 with\ua0 the\ua0 intentional accumulation\ua0 of\ua0 technical\ua0 debt\ua0 in\ua0 software\ua0 startups.\ua0 We interviewed 16 professionals involved in seven different software startups.\ua0 We\ua0 find\ua0 that\ua0 the\ua0 startup\ua0 phase,\ua0 the\ua0 experience\ua0 of\ua0 the developers,\ua0 software\ua0 knowledge\ua0 of\ua0 the\ua0 founders,\ua0 and\ua0 level\ua0 of employee\ua0 growth\ua0 are\ua0 some\ua0 of\ua0 the\ua0 organizational\ua0 factors\ua0 that influence\ua0 the\ua0 intentional\ua0 accumulation\ua0 of\ua0 technical\ua0 debt.\ua0 In addition,\ua0 we\ua0 find\ua0 the\ua0 software\ua0 startups\ua0 are\ua0 typically\ua0 driven\ua0 to achieve\ua0 a\ua0 “good\ua0 enough\ua0 level,”\ua0 and\ua0 this\ua0 guides\ua0 the\ua0 amount\ua0 of technical debt that they intentionally accumulate to balance the benefits\ua0 of\ua0 speed\ua0 to\ua0 market\ua0 and\ua0 reduced\ua0 resources\ua0 with\ua0 the challenges of later addressing technical debt

A Profession as Enterprise Architect

The present trend shows that Enterprise Architecture (EA) is an essential resource to improve the organizational efficiency, effectiveness, and agility, both in the business and the technology environment. The Enterprise Architect professionals who are working in this area are thus essential for operations in organizational transformation and development, therefore, vital to understand the ambition of this profession. There are several academic studies available concerning EA. However, there are few empirically based studies which in particularly reflect the Enterprise Architect profession. This study, examining the profession of the Enterprise Architect, sheds new light on what these professionals do within their organization on an every-day basis and how this view differs from how the profession is described in existing research. The purpose of this paper is to explore and compare how the Enterprise Architect profession is described both by academics and by empirically collected data. We perceive five topics that are essential to a comprehensive, rich picture of the profession; the role, competence, power, style of acting and main focus. The study is based on an initial literature survey and an empirically based study based on interviews with Enterprise Architects in ten large Swedish organizations. Our interviews show that the architect's work in several aspects is consistent with the literature but in other respects, an evident dissimilarity is revealed. One of the most obvious differences is the architect's mindset in terms of working in a reactive or a proactive way. Our interviews show that architects are working primarily in a reactive approach both in terms of how their roles are described but also in relation to how the EA function is set up. Although it is evidential that most of the architects’ work is based on a reactive basis, the architects claim it would be inappropriate with a purely proactive approach. Nevertheless, the establishment of the EA as a function within the interviewed organizations seems to have been well implemented, where architectural principles are determined as mandatory, while an interesting finding is that major or radical IT investments appears to overrule the architectural principles and is part of top management discretion only

An Empirical Investigation of the Harmfulness of Architectural Technical Debt

Background: In order to survive in today\u27s fast-growing and ever fast-changing business environments, large-scale software companies need to deliver customer value continuously, both from a short- and long-term perspective. However, the consequences of potential long-term and far-reaching negative effects of shortcuts and quick fixes made during the software development lifecycle, described as Technical Debt (TD), can impede the software development process.Objective: The overall goal of this Licentiate thesis is to empirically study and understand in what way and to what extent, TD in general and architectural TD specifically, influence today’s software development work and, specifically, with the intention of providing more quantitative insights into the field.Method: To achieve the objectives, a combination of both quantitative and qualitative research methodologies are used, including interviews, surveys, a systematic literature review, a longitudinal study, correlation analysis, and statistical tests. In five of the seven included studies, we use a combination of multiple research methods to achieve high validity.Results: We present results showing that software suffering from TD will cause various different negative effects on both the software and on the developing process. These negative effects can be illustrated from a technical, a financial and from a developer’s working situational perspective.Conclusion: This thesis contributes to the understanding and quantification of in what way and to what extent TD is harmful to software development organizations. The results show that software practitioners estimate that they waste 36% of their working time due to experiencing TD and that the TD is causing them to perform additional time-consuming work activities. This study also shows that, compared to all types of TD, architectural TD has the greatest negative impact on the daily software development work

Software Developer Productivity Loss Due to Technical Debt - A replication and extension study examining developers’ development work

Software companies need to deliver customer value continuously, both from a short- and long-term perspective. However, software development can be impeded by technical debt (TD). Although significant theoretical work has been undertaken to describe the negative effects of TD, little empirical evidence exists on how much wasted time and additional activities TD causes. The study aims to explore the consequences of TD in terms of wastage of development time. This study investigates on which activities this wasted time is spent and whether different TD types impact the wasted time differently. This study reports the results of a longitudinal study surveying 43 developers and including16 interviews followed by validation by an additional study using a different and independent dataset and focused on replicating the findings addressing the findings. The analysis of the reported wasted time revealed that developers waste, on average, 23% of their time due to TD and that developers are frequently forced to introduce new TD. The most common activity on which additional time is spent is performing additional testing. The study provides evidence that TD hinders developers by causing an excessive waste of working time, where the wasted time negatively affects productivity